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Using Data to Guide State Education Policy 
and Practice 


Executive Summary 

States and local school districts are building and im- 
proving data systems that track students, staffing, and 
expenditures. Significant amounts of data are now 
available to parents, educators, researchers, and poli- 
cymakers; however, the full promise of gathering and 
analyzing data to guide education policy and practice 
has yet to be realized. Although the technical aspects 
of the systems are largely in place, data analytics con- 
tinue to be used sparingly, mainly because they have 
limited utility for educators and policymakers. Even 
when data are analyzed to inform policy and practice, 
it is difficult for educators and policymakers to act on 
the results. The data largely sit in distinct silos, requir- 
ing individuals to jump through multiple hoops to ac- 
cess the information they need. 

It is up to governors to ensure that continued invest- 
ments in data systems are directed in ways that pro- 
mote the greater use of the data. Governors, chief state 
school officers, board members, legislators, and edu- 
cators can realize the promise of data to guide policy 
and practice if states: 

• Collect more actionable data designed to meet 
identified stakeholder questions, such as infor- 
mation on students’ mastery of standards, the 
impacts of academic interventions, and a clearer 
link between school and district expenditures 
and student performance; 

• Link multiple data systems through the adoption 
and use of common, open data standards; and 


• Provide new tools for aggregating and analyzing 
data that ease educators’ ability to offer individ- 
ualized instruction and support and policymak- 
ers’ ability to monitor performance. 

The Perils of Education Data 
to Date 

From test scores to teacher evaluations, data are 
everywhere in education. In fewer than 10 years, 
states and districts have taken bold steps to build 
and improve systems that collect data on students, 
staffing, and expenditures. States have gone from 
having little data about students or their perfor- 
mance to having an abundance of data on demo- 
graphics, attendance, standardized assessments, and 
graduation. The amount of data available to educa- 
tors, policymakers, and the public has increased ex- 
ponentially. Those systems were built to meet two 
goals: enable administrators and policymakers to 
make performance management decisions and guide 
educators in the personalization of instruction and 
support for students. 

Unfortunately, to date, the promise of using data to 
improve educational outcomes has not been fully met. 
Critical data remain elusive and obstacles impede pol- 
icymakers’ and educators’ ability to act on the data. 
Although in the past 10 years there has been a sea 
change in the availability of data — and in stakehold- 
ers’ attitudes regarding the use of data — the challenges 
have led many observers to characterize states as “data 
rich but information poor.” 
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Critical Data for Policymakers 
and Educators Are Not Available 

The federal government alone has spent nearly 
$500 million over the past five years to help states cre- 
ate sophisticated education data systems that can track 
the flow of individuals from early childhood through 
higher education and even into the workforce. When 
those systems were created, they were largely intended 
to serve as compliance mechanisms to track funding for 
programs and hold schools and districts accountable for 
performance. 1 Although large amounts of data were col- 
lected, often the specific data elements collected were in 
response to a reporting requirement rather than to help 
guide instruction or decisionmaking. For the most part, 
the systems facilitated the flow of data in one direction: 
up — from school to district to state to federal agency. 

States were particularly good at building such compli- 
ance-driven systems. In fact, in six years, the nation 
went from having no state meet the Data Quality Cam- 
paign’s 10 Essential Elements of Longitudinal Data 
System to 36 states meeting that standard. At the end of 
2011, all states had the data to answer policy questions 
about important issues such as enrollment, assessments, 
and graduation rates. Although the technical aspects of 
the systems are largely in place, data continue to be 
used sparingly, because critical elements are not col- 
lected. For instance, only 1 1 states annually match and 
share K-12 and workforce data, limiting policymak- 
ers’ ability to monitor the entire education pipeline. 2 

Because critical data elements are not yet collected 
and links between data systems are not in place, gov- 
ernors are still unable to answer these and other im- 
portant policy and program questions: 

• Which middle schools are most effective, as mea- 
sured by student transcripts, at preparing students 
to succeed in rigorous high school courses? 

• Which teachers are more productive with which 
types of students? 

• Which professional development programs have 
the greatest impact on the effectiveness of teach- 
ers, as measured by student performance? 


• What are the earnings of various types of col- 
lege graduates once they are in the workforce? 

The data systems that states have built thus far 
serve some useful but limited functions. They cap- 
ture operational data, which allow school and state 
leadership to track how many low-income students 
have dropped out. The data can also support orga- 
nizational decisions by, for instance, identifying 
which students would benefit from extra support. 
Yet significant gaps remain in the data collected. In 
fact, the McKinsey Global Initiative finds that the 
education sector ranks in the bottom 20 percent in 
its ability to harness the power of large data sets. 3 
States and districts do not collect data that capture 
how students behave and interact with academic re- 
sources. For instance, currently available data might 
tell a teacher who needs extra help, but the infor- 
mation does not provide insight on what is wrong 
academically or why the student may be struggling. 
Without access to that information, educators are 
limited in their ability to make decisions about indi- 
vidualized instruction and support. 

At the same time, state policymakers have little to 
no comparable information about school or district 
resource use. For example, most states do not have 
a clear idea of which school districts are most pro- 
ductive when it comes to spending education dol- 
lars. Only a few states, including Florida and Texas, 
currently have the capability to calculate return on 
investment (ROI) in education. Without those met- 
rics, policymakers and the public cannot determine 
whether education dollars are well spent within their 
states or benchmark their fiscal efficiency and ef- 
fectiveness across other states. A recent analysis by 
the Center for American Progress found that school 
districts with below-average productivity spent over 
$950 more per student than did above-average school 
districts. The total estimated cost of that inefficiency 
to the nation’s school systems is $175 billion per year. 4 
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Available Data Are Difficult to 
Access and Use 

Even when data are available, they are difficult for pol- 
icymakers and educators to access and use. Notwith- 
standing the great technical advances states have made 
with their data systems, the Data Quality Campaign 
has found that no state has taken all of the necessary 
actions to ensure that education data are actually used. 5 
In most states, there is no single education data sys- 
tem that collects information on students, educators, 
and expenditures across the education pipeline; rather, 
multiple systems exist that cover the areas of early 
childhood, K-12, and postsecondary education. Of- 
ten, those distinct data systems are stand-alone, pro- 
prietary applications from a particular vendor, which 


can make it difficult for the systems to interact. As a 
result, volumes of information are often isolated from 
each other in silos. 

Data sitting in silos leads to inefficient or incomplete 
reporting and inhibits their use as individuals struggle 
to obtain the information they need. In such cases, a 
student’s information can appear in many places, but 
the data fields may not be identical, or practitioners 
may have incomplete or inaccurate information. For 
instance, the insights gleaned from tutors in after- 
school programs cannot be shared with regular class- 
room teachers. The most extreme manifestation of the 
problem is between early childhood and K-12 systems, 
where kindergarten teachers cannot see what students 


Using Data to Monitor Progress 

Governors can use data as the basis for regular conversations with system heads to make policy decisions 
and improve their state’s performance on a variety of education metrics. For example, in Maryland, 
Governor Martin O’Malley uses his Delivery Unit, a component of the StateStat process, to systemati- 
cally improve performance. Ten state agencies regularly send in data that are analyzed and turned into 
briefing memos for the governor and his executive team. In education, this process has become kn own 
as StudentStat. Governor O’Malley has set a broad, strategic goal of improving student achievement and 
school, college, and career readiness by 25 percent in 2015. The K-12 state superintendent, Secretary 
of Higher Education, representatives of the public two-and four-year colleges, independent universities, 
and other key stakeholders, meet quarterly to review the 7 strategies and 3 1 data metrics associated with 
the strategic goal. Among the many targets the Delivery Unit tracks progress on are: 3 

• The number of students who enter kindergarten meeting the state’s Model for School Readiness 
Standards. Over the course of the unit’s discussing this effort, the percentage of students who meet 
the standards has risen from 67 in 2006-07 to 81 percent in 2010-11. As a consequence, the Deliv- 
ery Unit has set a new target of 92 percent fully ready by 2014-15. 

• The number of STEM degrees awarded at Maryland 2- and 4-year public and independent universi- 
ties. 

• The number of career and technical education (CTE) graduates earning industry certifications and / 
or licenses related to their program of study. 

“The full Maryland Education Delivery Unit plan can be accessed at http://www.statestat.marvland.gov/GDU/2EducationDelivervPlan.pdf . 
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have been doing in other programs to get ready for 
school. It is as if they are running a relay race, but 
there is no “data baton” to hand off to teachers in the 
next educational phase. 

States, districts, and schools lack a broader plat- 
form that would allow different data-analysis tools 
and software applications to be interoperable. 6 
In many cases, a teacher cannot go to a single portal 
for all the information needed on a student and must 
instead string together data from at least three different 
sources by logging on to a state system, an in-school 
grade book, and then a district-run site that contains 
links between data and learning tools. 

How States Can Achieve the 
Promise of Education Data 

If data systems and the analysis they permit live up to 
their promise, they could become a fundamental com- 
ponent of a well-functioning education system. Invest- 
ments in sophisticated longitudinal data systems can 
have long-term positive benefits. It is up to governors 
to ensure that the investments are directed in ways that 
promote the greater use of data. Governors, chief state 
school officers, board members, legislators, and edu- 
cators can realize the promise of data and analysis to 
guide policy and practice if states: 

• Collect more actionable data; 

• Link multiple data systems and design them to 
be interoperable; and 

• Create new tools for aggregating and analyzing 
data. 

Collect Actionable Data 

Data systems can be built to answer many of the ques- 
tions that parents, teachers, and policymakers ask. 
However, most education data systems were built to 
facilitate compliance rather than answer pertinent in- 
struction and policy questions. To ensure that the data 
available are useful, states should bring together edu- 
cators and policymakers to clearly define their data 
needs. Defining the “killer questions” that educators 


and policymakers seek to answer will help the indi- 
viduals managing the data systems target technology 
upgrades and make changes to data-collection guide- 
lines. To remain current and efficient, states should 
also annually evaluate the type and amount of data 
that educators and policymakers can access. 

Educators desire more information on the usefulness of 
particular instructional practices, programs, and sup- 
ports. For instance, only 12 and 18 states, respectively, 
collect attendance and discipline data daily. 7 Without 
that information, it is difficult for an administrator 
to monitor whether a dropout-prevention program is 
successful for particular students. The information is 
critical for forming the foundation of an early warning 
data system and for monitoring interventions. Simi- 
larly, information on students’ content mastery would 
help educators offer support to struggling students be- 
fore they fall off track to graduation. As student learn- 
ing shifts to using more digital tools and platforms, 
a flood of keystroke data from games, simulations, 
virtual environments, end-of-unit quizzes, and adap- 
tive assessments could be fed into a data system. The 
ability to learn about students in this way, and then 
use analytics to customize their learning, could be one 
of the most important education developments of the 
decade. 8 

Policymakers require more financial information on 
schools and districts for performance management de- 
cisions. Governors can gain more actionable financial 
information by requesting ROI data, which can lead 
to new conversations across the state about effective 
resource allocation. Those conversations can include 
determining the optimal size of school districts as well 
as ways to share services, find purchasing efficien- 
cies, and improve business operations in areas such as 
food service and transportation. When Delaware en- 
gaged in such conversations, the analysis showed that 
the state could save about 13 percent through shared 
services. 9 The Council of Great City Schools has de- 
veloped a list of indicators for districts to use across 
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functions such as human resources and technology. 
The council found that a typical urban district in the 
bottom quartile on those functions could save from 
$20 million to $50 million per year by moving closer 
to the median performance of their peers. 10 

The standardization and collection of financial data 
are critical for performance management. States can 
create a common financial reporting system for all 
schools, districts, and the education agency to use; 
Rhode Island recently did this through its Uniform 
Chart of Accounts. 11 Having a common method for 
categorizing expenditures will enable the state to more 
easily and accurately benchmark school and district 
expenditures and calculate productivity. The process 
of creating a standard financial reporting system will 
take time. Meantime, states can quickly begin to cal- 
culate ROI by requiring districts to disclose average 
per-pupil expenditures at the school level. Currently, 
the differences in expenditures from school to school 
(usually based on differences in actual teacher sala- 
ries) can vary by as much as $1 million per building. 
By showing the differences in spending in schools 
across districts rather than burying those differences in 
district-wide averages, states will be able to highlight 
spending inefficiencies in new ways. 

Link and Make Data Systems Interoperable 

Data from many different systems are necessary to 
make informed instruction and policy decisions. To 
take advantage of the data they collect, states need to 
design their systems to be interoperable (that is, able 
to integrate data from one system with data from a dif- 
ferent system) across agencies and institutions. Creat- 
ing interoperable data systems is important because it 
would allow education stakeholders to compare data; 
facilitate collaboration among state agencies, institu- 
tions of higher education, and school districts; and, 
ultimately, provide better answers to a broader set of 
policy and practice questions. For instance, in a state 
that has interoperable K-12 and postsecondary data 
systems, educators and policymakers can receive in- 
formation on the numbers of students enrolling in re- 


medial education in the state’s higher education sys- 
tem from a particular high school. That information 
may lead a teacher to redesign a writing curriculum 
or a policymaker to tighten graduation requirements. 

States can facilitate data sharing by adopting com- 
mon, open standards for data and assessment sys- 
tems. The Common Education Data Standards 
(CEDS) support exchange of a specified set of the 
most commonly used education data elements that 
track students’ transition between educational sec- 
tors and levels. 12 States will likely find value in shar- 
ing data with other states when the standards become 
more commonplace. For example, in areas with high 
levels of cross-state mobility, such as Kansas City, 
neighboring states may be interested in answering 
questions such as how former high school students 
from Kansas are performing in postsecondary edu- 
cation and/or the workforce in Missouri . 13 Partner- 
ships among states can help policymakers monitor 
the performance of their education and workforce 
systems and make policy changes to counteract any 
issues the data highlight. 

Implementing interoperability standards will gener- 
ate at least three benefits for states. First, doing so 
will provide incentives for vendors to develop appli- 
cations that generate new insights from data across 
systems that currently do not speak to each other. 
For example, an application could combine atten- 
dance data with academic and social services data to 
provide schools and districts with a much more so- 
phisticated understanding of who is dropping out of 
school and why. Second, states will not be locked in 
to a single application or a single vendor, a practice 
that can inhibit innovation and prevent states from 
enjoying the benefits of price competition among 
vendors. If tests are stored in a common format, it 
becomes easier for states to put their testing contract 
out to new bidders. Third, the common, open data 
standards will help reduce costs, especially on data 
integration, and will help ensure consistency across 
testing platforms. 
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Provide Easy-to-Use Tools for Data 
Aggregation and Analysis 

If data are to help inform day-to-day instruction and 
improve outcomes, states must provide a single por- 
tal in which test data can be integrated with teachers’ 
classroom data (such as instructional approaches and 
grades), along with micro-level data from embedded 
formative assessments and digital instructional tools. 
States can integrate all of that disparate information by 
building learning management systems (LMS) — plat- 
forms that aggregate data from multiple sources for 
easy access and use. 

An LMS offers several advantages. First, it operates 
as the foundation for linking the Common Core State 
Standards and the new assessments. In an LMS, teach- 
ers will be able to see a visual depiction of learning 
paths across all content areas as well as which lessons, 
digital learning tools, and test questions link to which 
standards. With the administration of online tests, the 
LMS will generate immediate scores for teachers and 
students to see. Second, the LMS will function as a 
warehouse for instructional resources and the delivery 
of job-embedded professional development. Teachers 
will be able to access videos of other highly effective 
teachers and model lesson plans created by master 
teachers. Third, principals will be able to use the LMS 
to add information for a teacher’s evaluation. States 
and districts will develop interactive reports demon- 
strating teacher effectiveness by grade, subject area 
taught, years of service, classroom and summative stu- 
dent test data, and graduate degree attainment. Finally, 
the LMS will contain tools that streamline the school 
and district improvement planning process and reduce 
all data reporting burdens. 

To date, large urban school districts have led most of 
the efforts to construct the first generation of LMSs. 
New York City’s Achievement Reporting and Innova- 
tion System (ARIS) allows teachers in inquiry teams to 


analyze student performance against the state standards 
and tailor their instruction accordingly . 14 Texas is an ex- 
ample of a state that is working to facilitate intrastate 
interoperability by developing a more flexible informa- 
tion system platform. The platform will offer smaller 
districts a shared, state-sponsored student information 
system and make it easy for districts with existing sys- 
tems to provide seamless data reporting. For example, 
districts will only enter attendance data once, reducing 
their reporting burdens. That information is then avail- 
able to teachers and administrators in real-time dash- 
board formats, where it helps to flag students with po- 
tential problems. If successful, the new system will not 
only reduce costs and streamline existing state account- 
ability processes but also equip educators with relevant 
information they can use to help their students . 15 

Beyond building new platforms, states can create tools 
that allow educators and policymakers to more eas- 
ily interpret data and make decisions. Colorado has 
developed a web-based portal (www.schoolview.org) 
that enables teachers to track individual students’ test 
scores over time and gauge their progress. Arkansas 
has developed a set of data portals for use by mul- 
tiple stakeholders, from parents to policymakers. Each 
stakeholder is granted a different level of access de- 
pending on his or her role. The state has also devel- 
oped Hive (http://hive.arkansas.gov) . an online com- 
munity that enables educators to construct and discuss 
visualizations using state data. 

Conclusion 

State governments face a productivity challenge in ed- 
ucation. They must improve performance so that more 
students are ready for college and careers than ever 
before, but do so in an era of flat revenues. Schools, 
districts, and states are under pressure to operate more 
efficiently and more effectively. To meet this chal- 
lenge, states need to harness the power of data to in- 
form decisions from the classroom to the statehouse. 

Contact: Ryan Reyna 
Program Director, Education Division 
rreyna @ nga.org 
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